Domain adaptation, adapting models from domains rich in labeled training datato domains poor in such data, is a fundamental NLP challenge. We introduce aneural network model that marries together ideas from two prominent strands ofresearch on domain adaptation through representation learning: structuralcorrespondence learning (SCL, (Blitzer et al., 2006)) and autoencoder neuralnetworks. Particularly, our model is a three-layer neural network that learnsto encode the nonpivot features of an input example into a low-dimensionalrepresentation, so that the existence of pivot features (features that areprominent in both domains and convey useful information for the NLP task) inthe example can be decoded from that representation. The low-dimensionalrepresentation is then employed in a learning algorithm for the task. Moreover,we show how to inject pre-trained word embeddings into our model in order toimprove generalization across examples with similar pivot features. On the taskof cross-domain product sentiment classification (Blitzer et al., 2007),consisting of 12 domain pairs, our model outperforms both the SCL and themarginalized stacked denoising autoencoder (MSDA, (Chen et al., 2012)) methodsby 3.77% and 2.17% respectively, on average across domain pairs.
展开▼
机译：NLP面临的基本挑战是，域自适应（将模型从富含标签的训练数据的域适应到缺乏此类数据的域）进行适应。我们介绍了一种神经网络模型，该模型将通过表示学习的两个主要领域研究思路融合在一起：结构对应学习（SCL，（Blitzer et al。，2006））和自动编码器神经网络。特别地，我们的模型是一个三层神经网络，该网络学习以将输入示例的非枢轴特征编码为低维表示，以便存在枢轴特征（在两个域中都突出并且可以为NLP任务传达有用信息的特征）在该示例中，可以根据该表示进行解码。然后将低维表示形式用于任务的学习算法。此外，我们展示了如何将预训练的词嵌入注入我们的模型中，以提高具有相似枢轴特征的示例之间的通用性。在包含12个域对的跨域产品情感分类（Blitzer等，2007）的任务上，我们的模型优于SCL和归一化堆叠式去噪自动编码器（MSDA，（Chen等，2012））方法，降低了3.77％和跨域对的平均值分别为2.17％。
展开▼
